You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by "Michael Dürig (JIRA)" <ji...@apache.org> on 2012/07/09 21:01:35 UTC
[jira] [Created] (OAK-175) MemoryNodeStateBuilder inefficient for
large child node lists
Michael Dürig created OAK-175:
---------------------------------
Summary: MemoryNodeStateBuilder inefficient for large child node lists
Key: OAK-175
URL: https://issues.apache.org/jira/browse/OAK-175
Project: Jackrabbit Oak
Issue Type: Improvement
Components: core
Reporter: Michael Dürig
OAK-174 leverages the {{getChildBuilder()}} method introduced with OAK-170. This revealed a performance issue for large child node list with the {{MemoryNodeStateBuilder}} implementation: without additional measures (see below) {{RootImplTest.largeChildList()}} is very slow.
The additional measures I currently implemented consist of counting the overall number of changes to a tree and purge these down to the branch when a certain threshold is reached. See {{RootImpl.purge()}}. However, I think this mechanism is too ad-hoc and we should find a more general solution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-175) MemoryNodeStateBuilder inefficient for
large child node lists
Posted by "Jukka Zitting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13409883#comment-13409883 ]
Jukka Zitting commented on OAK-175:
-----------------------------------
A part of this problem was that we didn't yet have an optimized compareAgainstBase() implementation for KernelNodeStates. I added a first draft in revision 1359413, which took 50% off from the largeChildList() test time.
> MemoryNodeStateBuilder inefficient for large child node lists
> -------------------------------------------------------------
>
> Key: OAK-175
> URL: https://issues.apache.org/jira/browse/OAK-175
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
> Reporter: Michael Dürig
>
> OAK-174 leverages the {{getChildBuilder()}} method introduced with OAK-170. This revealed a performance issue for large child node list with the {{MemoryNodeStateBuilder}} implementation: without additional measures (see below) {{RootImplTest.largeChildList()}} is very slow.
> The additional measures I currently implemented consist of counting the overall number of changes to a tree and purge these down to the branch when a certain threshold is reached. See {{RootImpl.purge()}}. However, I think this mechanism is too ad-hoc and we should find a more general solution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-175) MemoryNodeStateBuilder inefficient for
large child node lists
Posted by "Michael Dürig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13411392#comment-13411392 ]
Michael Dürig commented on OAK-175:
-----------------------------------
One way to handle the remaining problem is to push the purge logic down to the {{NodeStore}} instead of handling it explicitly in {{RootImpl}}. A caching {{NodeStore}} implementation (OAK-167) might take care of this.
> MemoryNodeStateBuilder inefficient for large child node lists
> -------------------------------------------------------------
>
> Key: OAK-175
> URL: https://issues.apache.org/jira/browse/OAK-175
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
> Reporter: Michael Dürig
>
> OAK-174 leverages the {{getChildBuilder()}} method introduced with OAK-170. This revealed a performance issue for large child node list with the {{MemoryNodeStateBuilder}} implementation: without additional measures (see below) {{RootImplTest.largeChildList()}} is very slow.
> The additional measures I currently implemented consist of counting the overall number of changes to a tree and purge these down to the branch when a certain threshold is reached. See {{RootImpl.purge()}}. However, I think this mechanism is too ad-hoc and we should find a more general solution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (OAK-175) MemoryNodeStateBuilder inefficient for
large child node lists
Posted by "Michael Dürig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/OAK-175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13410252#comment-13410252 ]
Michael Dürig commented on OAK-175:
-----------------------------------
Thanks Jukka, that's great stuff. It only solves half the problem though.
The other half is cause by {{MemoryNodeStateBuilder.snapshot}} which deeply copies all child nodes. When adding node sequentially this leads to a quadratic performance degradation in the number of child nodes. The solution I implemented through {{RootImpl.purge()}} purges changes down to the branch when the number of changes kept in memory reach a certain threshold. While this works, I find my approach too ad-hoc and would like to come up with a more general solution.
> MemoryNodeStateBuilder inefficient for large child node lists
> -------------------------------------------------------------
>
> Key: OAK-175
> URL: https://issues.apache.org/jira/browse/OAK-175
> Project: Jackrabbit Oak
> Issue Type: Improvement
> Components: core
> Reporter: Michael Dürig
>
> OAK-174 leverages the {{getChildBuilder()}} method introduced with OAK-170. This revealed a performance issue for large child node list with the {{MemoryNodeStateBuilder}} implementation: without additional measures (see below) {{RootImplTest.largeChildList()}} is very slow.
> The additional measures I currently implemented consist of counting the overall number of changes to a tree and purge these down to the branch when a certain threshold is reached. See {{RootImpl.purge()}}. However, I think this mechanism is too ad-hoc and we should find a more general solution.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira