You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@jackrabbit.apache.org by "Julian Reschke (Created) (JIRA)" <ji...@apache.org> on 2011/12/19 14:27:30 UTC
[jira] [Created] (JCR-3185) refactor consistency checks in
BundleDBPersistenceManager into a standalone class that could be re-used
for other PMs
refactor consistency checks in BundleDBPersistenceManager into a standalone class that could be re-used for other PMs
---------------------------------------------------------------------------------------------------------------------
Key: JCR-3185
URL: https://issues.apache.org/jira/browse/JCR-3185
Project: Jackrabbit Content Repository
Issue Type: Task
Components: jackrabbit-core
Reporter: Julian Reschke
Assignee: Julian Reschke
Priority: Minor
see subject
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3185) refactor consistency checks in
BundleDBPersistenceManager into a standalone class that could be re-used
for other PMs
Posted by "Julian Reschke (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173329#comment-13173329 ]
Julian Reschke commented on JCR-3185:
-------------------------------------
I agree that returning a collection/set/or list would make sense. However, if we ever change the code to use the paging feature we'll need the getCount() method anyway...
> refactor consistency checks in BundleDBPersistenceManager into a standalone class that could be re-used for other PMs
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: JCR-3185
> URL: https://issues.apache.org/jira/browse/JCR-3185
> Project: Jackrabbit Content Repository
> Issue Type: Task
> Components: jackrabbit-core
> Reporter: Julian Reschke
> Assignee: Julian Reschke
> Priority: Minor
> Attachments: JCR-3185.diff
>
>
> see subject
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (JCR-3185) refactor consistency checks in
BundleDBPersistenceManager into a standalone class that could be re-used
for other PMs
Posted by "Julian Reschke (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julian Reschke updated JCR-3185:
--------------------------------
Attachment: JCR-3185.diff
Proposal.
(We could add the code to the AbstractBundlePM as well, but I wanted to keep it separate)
> refactor consistency checks in BundleDBPersistenceManager into a standalone class that could be re-used for other PMs
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: JCR-3185
> URL: https://issues.apache.org/jira/browse/JCR-3185
> Project: Jackrabbit Content Repository
> Issue Type: Task
> Components: jackrabbit-core
> Reporter: Julian Reschke
> Assignee: Julian Reschke
> Priority: Minor
> Attachments: JCR-3185.diff
>
>
> see subject
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3185) refactor consistency checks in
BundleDBPersistenceManager into a standalone class that could be re-used
for other PMs
Posted by "Jukka Zitting (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180372#comment-13180372 ]
Jukka Zitting commented on JCR-3185:
------------------------------------
> Loading the whole list in memory will run out of memory in many cases.
That's best handled on the client side, e.g. by making the consistency checker process things in chunks of say 1m nodes at a time. I'd track that as a separate issue.
> don't log the size
That's what I'd do.
> refactor consistency checks in BundleDBPersistenceManager into a standalone class that could be re-used for other PMs
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: JCR-3185
> URL: https://issues.apache.org/jira/browse/JCR-3185
> Project: Jackrabbit Content Repository
> Issue Type: Task
> Components: jackrabbit-core
> Reporter: Julian Reschke
> Assignee: Julian Reschke
> Priority: Minor
> Attachments: JCR-3185.diff
>
>
> see subject
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (JCR-3185) refactor consistency checks in
BundleDBPersistenceManager into a standalone class that could be re-used
for other PMs
Posted by "Julian Reschke (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julian Reschke resolved JCR-3185.
---------------------------------
Resolution: Fixed
> refactor consistency checks in BundleDBPersistenceManager into a standalone class that could be re-used for other PMs
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: JCR-3185
> URL: https://issues.apache.org/jira/browse/JCR-3185
> Project: Jackrabbit Content Repository
> Issue Type: Task
> Components: jackrabbit-core
> Reporter: Julian Reschke
> Assignee: Julian Reschke
> Priority: Minor
> Attachments: JCR-3185.diff
>
>
> see subject
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3185) refactor consistency checks in
BundleDBPersistenceManager into a standalone class that could be re-used
for other PMs
Posted by "Julian Reschke (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173348#comment-13173348 ]
Julian Reschke commented on JCR-3185:
-------------------------------------
OK; so let's use a list for now, do not add getNumberOfBundles(), and address this in the future when we actually use paging,
> refactor consistency checks in BundleDBPersistenceManager into a standalone class that could be re-used for other PMs
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: JCR-3185
> URL: https://issues.apache.org/jira/browse/JCR-3185
> Project: Jackrabbit Content Repository
> Issue Type: Task
> Components: jackrabbit-core
> Reporter: Julian Reschke
> Assignee: Julian Reschke
> Priority: Minor
> Attachments: JCR-3185.diff
>
>
> see subject
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3185) refactor consistency checks in
BundleDBPersistenceManager into a standalone class that could be re-used
for other PMs
Posted by "Jukka Zitting (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173337#comment-13173337 ]
Jukka Zitting commented on JCR-3185:
------------------------------------
IIRC the paging feature in getAllNodeIds() was just added just so a client could iterate over all the nodes in chunks of predefined size. At least in that use case the total number of nodes was not needed.
> refactor consistency checks in BundleDBPersistenceManager into a standalone class that could be re-used for other PMs
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: JCR-3185
> URL: https://issues.apache.org/jira/browse/JCR-3185
> Project: Jackrabbit Content Repository
> Issue Type: Task
> Components: jackrabbit-core
> Reporter: Julian Reschke
> Assignee: Julian Reschke
> Priority: Minor
> Attachments: JCR-3185.diff
>
>
> see subject
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (JCR-3185) refactor consistency checks in
BundleDBPersistenceManager into a standalone class that could be re-used
for other PMs
Posted by "Jukka Zitting (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jukka Zitting updated JCR-3185:
-------------------------------
Fix Version/s: 2.3.7
Issue Type: Improvement (was: Task)
> refactor consistency checks in BundleDBPersistenceManager into a standalone class that could be re-used for other PMs
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: JCR-3185
> URL: https://issues.apache.org/jira/browse/JCR-3185
> Project: Jackrabbit Content Repository
> Issue Type: Improvement
> Components: jackrabbit-core
> Reporter: Julian Reschke
> Assignee: Julian Reschke
> Priority: Minor
> Fix For: 2.3.7
>
> Attachments: JCR-3185.diff
>
>
> see subject
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (JCR-3185) refactor consistency checks in
BundleDBPersistenceManager into a standalone class that could be re-used
for other PMs
Posted by "Julian Reschke (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julian Reschke updated JCR-3185:
--------------------------------
Attachment: (was: JCR-3185.diff)
> refactor consistency checks in BundleDBPersistenceManager into a standalone class that could be re-used for other PMs
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: JCR-3185
> URL: https://issues.apache.org/jira/browse/JCR-3185
> Project: Jackrabbit Content Repository
> Issue Type: Task
> Components: jackrabbit-core
> Reporter: Julian Reschke
> Assignee: Julian Reschke
> Priority: Minor
> Attachments: JCR-3185.diff
>
>
> see subject
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (JCR-3185) refactor consistency checks in
BundleDBPersistenceManager into a standalone class that could be re-used
for other PMs
Posted by "Julian Reschke (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Julian Reschke updated JCR-3185:
--------------------------------
Attachment: JCR-3185.diff
Proposed change.
> refactor consistency checks in BundleDBPersistenceManager into a standalone class that could be re-used for other PMs
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: JCR-3185
> URL: https://issues.apache.org/jira/browse/JCR-3185
> Project: Jackrabbit Content Repository
> Issue Type: Task
> Components: jackrabbit-core
> Reporter: Julian Reschke
> Assignee: Julian Reschke
> Priority: Minor
> Attachments: JCR-3185.diff
>
>
> see subject
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3185) refactor consistency checks in
BundleDBPersistenceManager into a standalone class that could be re-used
for other PMs
Posted by "Thomas Mueller (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180351#comment-13180351 ]
Thomas Mueller commented on JCR-3185:
-------------------------------------
Loading the whole list in memory will run out of memory in many cases.
The call isn't supposed to block other operations (specially for DataStore garbage collection).
Many persistence manager implementations can't return the exact size() (specially, but not only because the call is non-blocking).
Therefore the easiest solution is to revert the change, and don't log the size. Or add a method getNumberOfBundlesEstimate() or so.
> refactor consistency checks in BundleDBPersistenceManager into a standalone class that could be re-used for other PMs
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: JCR-3185
> URL: https://issues.apache.org/jira/browse/JCR-3185
> Project: Jackrabbit Content Repository
> Issue Type: Task
> Components: jackrabbit-core
> Reporter: Julian Reschke
> Assignee: Julian Reschke
> Priority: Minor
> Attachments: JCR-3185.diff
>
>
> see subject
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (JCR-3185) refactor consistency checks in
BundleDBPersistenceManager into a standalone class that could be re-used
for other PMs
Posted by "Jukka Zitting (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JCR-3185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13173312#comment-13173312 ]
Jukka Zitting commented on JCR-3185:
------------------------------------
+1 Looks straightforward enough, and moving the code outside the PM implementations make sense.
I wonder if it might be a good idea to change getAllNodeIds() to return a Collection (or a Set) instead of an Iterable so we wouldn't need the extra getNumberOfBundles() call. AFAIUI all PM implementations in any case implement getAllNodeIds() with an in-memory list so the memory impact shouldn't be a problem (and if it is, the getAllNodeIds() signature already contains the maxCount option for dealing with that issue). And implementing getAllNodeIds() with an actual backend iterator opens up problems about when and how the resources used by the iterator can and should be released.
PS. There's an svn:executable property in the patch that probably shouldn't be there.
> refactor consistency checks in BundleDBPersistenceManager into a standalone class that could be re-used for other PMs
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: JCR-3185
> URL: https://issues.apache.org/jira/browse/JCR-3185
> Project: Jackrabbit Content Repository
> Issue Type: Task
> Components: jackrabbit-core
> Reporter: Julian Reschke
> Assignee: Julian Reschke
> Priority: Minor
> Attachments: JCR-3185.diff
>
>
> see subject
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira