You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by "Alex Parvulescu (JIRA)" <ji...@apache.org> on 2012/12/04 16:54:59 UTC
[jira] [Created] (JCR-3477) Lazy query result sets
Alex Parvulescu created JCR-3477:
------------------------------------
Summary: Lazy query result sets
Key: JCR-3477
URL: https://issues.apache.org/jira/browse/JCR-3477
Project: Jackrabbit Content Repository
Issue Type: Improvement
Components: jackrabbit-core
Reporter: Alex Parvulescu
Priority: Minor
Following JCR-3472, I'd like to open a discussion on what can be reasonably expected from a query that does/can do lazy fetching of results.
Creating ~110k nodes and fetching them all looks like it goes in the vicinity of 400mb of memory, so the default 256mb setup for tests is not sufficient to run such a test, as we've already seen on JCR-3472.
What are the expectations around lazy loading and queries that can potentially hit a high number of nodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3477) Lazy query result sets
Posted by "Michael Dürig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13509885#comment-13509885 ]
Michael Dürig commented on JCR-3477:
------------------------------------
I encountered this issue while investigating the root cause for JCR-3473. The problem already occurs on running the query (no need to fetch the result rows/nodes). I wonder what the original expectation for queries was. The presence of LargeResultSetTest leads me to believe that such queries should actually work but unfortunately JCR-3472 prevented us from detecting they don't. If OTOH we don't except these queries to work, we need to find a way to fix JCR-3473 without having to rely on queries.
> Lazy query result sets
> -----------------------
>
> Key: JCR-3477
> URL: https://issues.apache.org/jira/browse/JCR-3477
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
>
> Following JCR-3472, I'd like to open a discussion on what can be reasonably expected from a query that does/can do lazy fetching of results.
> Creating ~110k nodes and fetching them all looks like it goes in the vicinity of 400mb of memory, so the default 256mb setup for tests is not sufficient to run such a test, as we've already seen on JCR-3472.
> What are the expectations around lazy loading and queries that can potentially hit a high number of nodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3477) Lazy query result sets
Posted by "Marcel Reutegger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510340#comment-13510340 ]
Marcel Reutegger commented on JCR-3477:
---------------------------------------
At least my expectation is that Jackrabbit should be able to handle large result sets with reasonable memory usage. Do we have more details on what exactly consumes that much memory? Is it specific to some queries or is it for any kind of query?
> Lazy query result sets
> -----------------------
>
> Key: JCR-3477
> URL: https://issues.apache.org/jira/browse/JCR-3477
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
>
> Following JCR-3472, I'd like to open a discussion on what can be reasonably expected from a query that does/can do lazy fetching of results.
> Creating ~110k nodes and fetching them all looks like it goes in the vicinity of 400mb of memory, so the default 256mb setup for tests is not sufficient to run such a test, as we've already seen on JCR-3472.
> What are the expectations around lazy loading and queries that can potentially hit a high number of nodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3477) Lazy query result sets
Posted by "Marcel Reutegger (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510422#comment-13510422 ]
Marcel Reutegger commented on JCR-3477:
---------------------------------------
IIRC, these kind of queries are special cased because it basically means traversing the hierarchy. Maybe something is wrong there...
> Lazy query result sets
> -----------------------
>
> Key: JCR-3477
> URL: https://issues.apache.org/jira/browse/JCR-3477
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
>
> Following JCR-3472, I'd like to open a discussion on what can be reasonably expected from a query that does/can do lazy fetching of results.
> Creating ~110k nodes and fetching them all looks like it goes in the vicinity of 400mb of memory, so the default 256mb setup for tests is not sufficient to run such a test, as we've already seen on JCR-3472.
> What are the expectations around lazy loading and queries that can potentially hit a high number of nodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (JCR-3477) Lazy query result sets
Posted by "Michael Dürig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510373#comment-13510373 ]
Michael Dürig edited comment on JCR-3477 at 12/5/12 9:21 AM:
-------------------------------------------------------------
I have so far seen it for queries of type
/jcr:root/root//element(*)
when the tree rooted at "root" is large (no flat hierarchies involved). I.e. ~110k nodes bumps memory usage up to 400MB on Query.execute().
was (Author: mduerig):
I have so far seen it for queries of type
/jcr:root/root//element(*)
when the tree rooted at "root" is large. I.e. ~110k nodes bumps memory usage up to 400MB on Query.execute().
> Lazy query result sets
> -----------------------
>
> Key: JCR-3477
> URL: https://issues.apache.org/jira/browse/JCR-3477
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
>
> Following JCR-3472, I'd like to open a discussion on what can be reasonably expected from a query that does/can do lazy fetching of results.
> Creating ~110k nodes and fetching them all looks like it goes in the vicinity of 400mb of memory, so the default 256mb setup for tests is not sufficient to run such a test, as we've already seen on JCR-3472.
> What are the expectations around lazy loading and queries that can potentially hit a high number of nodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3477) Lazy query result sets
Posted by "Michael Dürig (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510373#comment-13510373 ]
Michael Dürig commented on JCR-3477:
------------------------------------
I have so far seen it for queries of type
/jcr:root/root//element(*)
when the tree rooted at "root" is large. I.e. ~110k nodes bumps memory usage up to 400MB on Query.execute().
> Lazy query result sets
> -----------------------
>
> Key: JCR-3477
> URL: https://issues.apache.org/jira/browse/JCR-3477
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Alex Parvulescu
> Priority: Minor
>
> Following JCR-3472, I'd like to open a discussion on what can be reasonably expected from a query that does/can do lazy fetching of results.
> Creating ~110k nodes and fetching them all looks like it goes in the vicinity of 400mb of memory, so the default 256mb setup for tests is not sufficient to run such a test, as we've already seen on JCR-3472.
> What are the expectations around lazy loading and queries that can potentially hit a high number of nodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira