You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jackrabbit.apache.org by Fabián Mandelbaum <fm...@gmail.com> on 2018/03/15 09:47:12 UTC
Version history of deleted nodes and garbage collection
Hello,
my application based on JackRabbit 2.x (albeit an old .x, IIRC, .6, still
cannot upgrade to latest because it breaks my build... lots of dependencies
from other libs/frameworks I use) uses version history tracking of items
stored under certain paths.
One of the features of the application is to be able to erase the items (wipe
them, and all related nodes and stored information forever) in question.
When deleting those items, what happens with their version history tree? Is
the version history tree eligible for garbage collection (ran by a "robot"
in my app once a day)? Or should I also delete the version history tree "by
hand" (which poses lots of problems with session state... I have to do
session.save() 'in the middle' of the deletion process)?
Thanks in advance for your answers.
Kind regards.
--
Fabián Mandelbaum
IS Engineer
Re: Version history of deleted nodes and garbage collection
Posted by Fabián Mandelbaum <fm...@gmail.com>.
Yes, those are the ones that might be left orphan (pointing to no existing
versionable node anymore) in my app. Too complex to explain here...
sorry... but the way we store data and their relationships - in process of
being reworked - and some not-very-elegant code did leave some such nodes
(cannot tell how many, cuz in your original answer you've told me there
might be no way to 'write such a query' to know who the orphans are).
I know they (the orphaned revision history nodes) should not be accessible
anymore... but I'd like to have them removed programatically.
Thanks again.
Best regards, Fabián.
On Fri, Mar 16, 2018 at 9:39 AM, Julian Reschke <ju...@gmx.de>
wrote:
> On 2018-03-16 13:32, Fabián Mandelbaum wrote:
>
>> Hello, the saga continues... I've wrote a small robot to traverse the
>> versions storage and find the unreferenced version histories (actually, it
>> finds all nt:versionHistory nodes whose jcr:versionableUuid point to a
>> non-stored-anymore node).
>> Using an old-but-still-working-at-least-with-JR-2.6 Java GUI tool to
>> browse and manipulate the repository, I've deleted some versionable nodes
>> to create the orphans.
>>
>> The full code (minus the copyright) for the robot is this:
>>
>> package com.calenco.core.robot;
>>
>> import com.calenco.storage.PooledRepoManager;
>> import java.util.TimerTask;
>> import javax.jcr.ItemNotFoundException;
>> import javax.jcr.Node;
>> import javax.jcr.NodeIterator;
>> import javax.jcr.RepositoryException;
>> import javax.jcr.Session;
>> import org.apache.jackrabbit.JcrConstants;
>> import org.apache.log4j.Logger;
>>
>> /**
>> *
>> * Task that will periodically check orphaned revision history nodes
>> * (nt:versionHistory) on the jcr:versionStorage tree and delete the
>> orphans
>> * found. Ideally, it should not find any orphan... but V3.1 and previous
>> * Calenco versions left some orphaned revision history nodes on certain
>> Eraser
>> * extension operations.
>> *
>> * @author fabman
>> */
>> public class JcrHistoryCleanerTimerTask extends TimerTask {
>>
>> public static final Logger LOG = Logger.getLogger(JcrHistoryCle
>> anerTimerTask.class);
>> private static final PooledRepoManager REPOMGR =
>> PooledRepoManager.getInstance();
>>
>> public JcrHistoryCleanerTimerTask() {
>> super();
>> }
>>
>> @Override
>> public void run() {
>> LOG.info(String.format("%s running...", getClass()));
>> Session nsession = null;
>> try {
>> nsession = REPOMGR.getSessionRW();
>> processNode(nsession, nsession.getNode("/jcr:system/
>> jcr:versionStorage"));
>> if (nsession.hasPendingChanges()) {
>> nsession.save();
>> }
>> } catch (RepositoryException ex) {
>> LOG.warn(String.format("Got %s while trying to clean
>> revision history tree", ex), ex);
>> } finally {
>> REPOMGR.releaseSessionRW(nsession);
>> }
>> }
>>
>> private void processNode(Session nsession, Node node) throws
>> RepositoryException {
>> String npath = node.getPath();
>> String ntype = node.getPrimaryNodeType().getName();
>> LOG.info(String.format("Processing node %s [%s]...", npath,
>> ntype));
>> if (null == ntype) {
>> LOG.warn(String.format("!!! Skipping node %s of type %s",
>> npath, ntype));
>> } else {
>> switch (ntype) {
>> case "rep:versionStorage":
>> // recurse...
>> NodeIterator nit = node.getNodes();
>> while (nit.hasNext()) {
>> processNode(nsession, nit.nextNode());
>> }
>> break;
>> case JcrConstants.NT_VERSIONHISTORY:
>> String vuuid = node.getProperty(JcrConstants.
>> JCR_VERSIONABLEUUID).getString();
>> if (!nodeExists(nsession, vuuid)) {
>> node.remove();
>> LOG.info(String.format(">>> Node with UUID %s is
>> not stored anymore... its revision history has been deleted...", vuuid));
>> }
>> break;
>> default:
>> LOG.warn(String.format("!!! Skipping node %s of type
>> %s", npath, ntype));
>> break;
>> }
>> }
>> }
>>
>> private boolean nodeExists(Session nsession, String uuid) {
>> try {
>> nsession.getNodeByIdentifier(uuid);
>> return true;
>> } catch (ItemNotFoundException ignoredReturningFalse) {
>> return false;
>> } catch (RepositoryException ex) {
>> LOG.warn(String.format("!!! Got unexpected %s while checking
>> if node with UUID %s exists", ex.getMessage(), uuid));
>> return false;
>> }
>> }
>> }
>>
>> When the robot is run, I get this on the log:
>>
>> 2018-03-16 08:43:06,444INFO com.calenco.core.robot.JcrHistoryCleanerTimerTask:42
>> - class com.calenco.core.robot.JcrHistoryCleanerTimerTask running...
>>
>> 2018-03-16 08:43:06,445INFO com.calenco.core.robot.JcrHistoryCleanerTimerTask:60
>> - Processing node /jcr:system/jcr:versionStorage [rep:versionStorage]...
>>
>> 2018-03-16 08:43:06,448INFO com.calenco.core.robot.JcrHistoryCleanerTimerTask:60
>> - Processing node /jcr:system/jcr:versionStorage/78
>> [rep:versionStorage]...
>>
>> 2018-03-16 08:43:06,449INFO com.calenco.core.robot.JcrHistoryCleanerTimerTask:60
>> - Processing node /jcr:system/jcr:versionStorage/78/11
>> [rep:versionStorage]...
>>
>> 2018-03-16 08:43:06,451INFO com.calenco.core.robot.JcrHistoryCleanerTimerTask:60
>> - Processing node /jcr:system/jcr:versionStorage/78/11/fe
>> [rep:versionStorage]...
>>
>> 2018-03-16 08:43:06,454INFO com.calenco.core.robot.JcrHistoryCleanerTimerTask:60
>> - Processing node /jcr:system/jcr:versionStorage
>> /78/11/fe/7811fe34-0aae-432a-9015-2626ee77bced [nt:versionHistory]...
>>
>> 2018-03-16 08:43:06,457INFO com.calenco.core.robot.JcrHistoryCleanerTimerTask:60
>> - Processing node /jcr:system/jcr:versionStorage/78/da
>> [rep:versionStorage]...
>>
>> 2018-03-16 08:43:06,459INFO com.calenco.core.robot.JcrHistoryCleanerTimerTask:60
>> - Processing node /jcr:system/jcr:versionStorage/78/da/be
>> [rep:versionStorage]...
>>
>> 2018-03-16 08:43:06,460INFO com.calenco.core.robot.JcrHistoryCleanerTimerTask:60
>> - Processing node /jcr:system/jcr:versionStorage
>> /78/da/be/78dabe1c-ead8-414d-8455-4ece9412c1a6 [nt:versionHistory]...
>>
>>
>> [LOTS_OF_RECURSIVE_TRAVERSAL_RELATED_STUFF_REMOVED_FROM_HERE]
>>
>>
>> 2018-03-16 08:43:06,737INFO com.calenco.core.robot.JcrHistoryCleanerTimerTask:60
>> - Processing node /jcr:system/jcr:versionStorage/e9
>> [rep:versionStorage]...
>>
>> 2018-03-16 08:43:06,739INFO com.calenco.core.robot.JcrHistoryCleanerTimerTask:60
>> - Processing node /jcr:system/jcr:versionStorage/e9/07
>> [rep:versionStorage]...
>>
>> 2018-03-16 08:43:06,739INFO com.calenco.core.robot.JcrHistoryCleanerTimerTask:60
>> - Processing node /jcr:system/jcr:versionStorage/e9/07/ed
>> [rep:versionStorage]...
>>
>> 2018-03-16 08:43:06,740INFO com.calenco.core.robot.JcrHistoryCleanerTimerTask:60
>> - Processing node /jcr:system/jcr:versionStorage
>> /e9/07/ed/e907edad-ad75-4572-ba44-56a9f5741fb9 [nt:versionHistory]...
>>
>> 2018-03-16 08:43:06,743WARN com.calenco.core.robot.JcrHistoryCleanerTimerTask:51
>> - Got javax.jcr.nodetype.ConstraintViolationException: Unable to perform
>> operation. Node is protected. while trying to clean revision history tree
>>
>> javax.jcr.nodetype.ConstraintViolationException: Unable to perform
>> operation. Node is protected.
>>
>> at org.apache.jackrabbit.core.ItemValidator.checkCondition(Item
>> Validator.java:276)
>>
>> at org.apache.jackrabbit.core.ItemValidator.checkRemove(ItemVal
>> idator.java:254)
>>
>> at org.apache.jackrabbit.core.ItemRemoveOperation.perform(ItemR
>> emoveOperation.java:63)
>>
>> at org.apache.jackrabbit.core.session.SessionState.perform(Sess
>> ionState.java:200)
>>
>> at org.apache.jackrabbit.core.ItemImpl.perform(ItemImpl.java:91)
>>
>> at org.apache.jackrabbit.core.ItemImpl.remove(ItemImpl.java:322)
>>
>> at com.calenco.core.robot.JcrHistoryCleanerTimerTask.processNod
>> e(JcrHistoryCleanerTimerTask.java:75)
>>
>> at com.calenco.core.robot.JcrHistoryCleanerTimerTask.processNod
>> e(JcrHistoryCleanerTimerTask.java:69)
>>
>> at com.calenco.core.robot.JcrHistoryCleanerTimerTask.processNod
>> e(JcrHistoryCleanerTimerTask.java:69)
>>
>> at com.calenco.core.robot.JcrHistoryCleanerTimerTask.processNod
>> e(JcrHistoryCleanerTimerTask.java:69)
>>
>> at com.calenco.core.robot.JcrHistoryCleanerTimerTask.processNod
>> e(JcrHistoryCleanerTimerTask.java:69)
>>
>> at com.calenco.core.robot.JcrHistoryCleanerTimerTask.run(JcrHis
>> toryCleanerTimerTask.java:46)
>>
>> at java.util.TimerThread.mainLoop(Timer.java:555)
>>
>> at java.util.TimerThread.run(Timer.java:505)
>>
>>
>> I've removed lots of log entries for nodes that don't need processing so
>> you don't have to scroll down that much to get to the interesting part. The
>> removed entries are basically the recursive traversal of the revision
>> history tree nodes. Version storage for node with UUID
>> e907edad-ad75-4572-ba44-56a9f5741fb9 is an orphan, there's no such node
>> stored anymore.
>>
>> So, it seems that (soundly, but won't let me remove what I need) the tree
>> under /jcr:system/jcr:versionStorage is protected in a way that some of the
>> "regular" node manipulation API (node.remove()) cannot be used on those
>> nodes.
>> ...
>>
>
> What type of node are you trying to delete? The version history node?
>
> Best regards, Julian
>
--
Fabián Mandelbaum
IS Engineer
Re: Version history of deleted nodes and garbage collection
Posted by Julian Reschke <ju...@gmx.de>.
On 2018-03-16 13:32, Fabián Mandelbaum wrote:
> Hello, the saga continues... I've wrote a small robot to traverse the
> versions storage and find the unreferenced version histories (actually,
> it finds all nt:versionHistory nodes whose jcr:versionableUuid point to
> a non-stored-anymore node).
> Using an old-but-still-working-at-least-with-JR-2.6 Java GUI tool to
> browse and manipulate the repository, I've deleted some versionable
> nodes to create the orphans.
>
> The full code (minus the copyright) for the robot is this:
>
> package com.calenco.core.robot;
>
> import com.calenco.storage.PooledRepoManager;
> import java.util.TimerTask;
> import javax.jcr.ItemNotFoundException;
> import javax.jcr.Node;
> import javax.jcr.NodeIterator;
> import javax.jcr.RepositoryException;
> import javax.jcr.Session;
> import org.apache.jackrabbit.JcrConstants;
> import org.apache.log4j.Logger;
>
> /**
> *
> * Task that will periodically check orphaned revision history nodes
> * (nt:versionHistory) on the jcr:versionStorage tree and delete the
> orphans
> * found. Ideally, it should not find any orphan... but V3.1 and previous
> * Calenco versions left some orphaned revision history nodes on
> certain Eraser
> * extension operations.
> *
> * @author fabman
> */
> public class JcrHistoryCleanerTimerTask extends TimerTask {
>
> public static final Logger LOG =
> Logger.getLogger(JcrHistoryCleanerTimerTask.class);
> private static final PooledRepoManager REPOMGR =
> PooledRepoManager.getInstance();
>
> public JcrHistoryCleanerTimerTask() {
> super();
> }
>
> @Override
> public void run() {
> LOG.info(String.format("%s running...", getClass()));
> Session nsession = null;
> try {
> nsession = REPOMGR.getSessionRW();
> processNode(nsession,
> nsession.getNode("/jcr:system/jcr:versionStorage"));
> if (nsession.hasPendingChanges()) {
> nsession.save();
> }
> } catch (RepositoryException ex) {
> LOG.warn(String.format("Got %s while trying to clean
> revision history tree", ex), ex);
> } finally {
> REPOMGR.releaseSessionRW(nsession);
> }
> }
>
> private void processNode(Session nsession, Node node) throws
> RepositoryException {
> String npath = node.getPath();
> String ntype = node.getPrimaryNodeType().getName();
> LOG.info(String.format("Processing node %s [%s]...", npath,
> ntype));
> if (null == ntype) {
> LOG.warn(String.format("!!! Skipping node %s of type %s",
> npath, ntype));
> } else {
> switch (ntype) {
> case "rep:versionStorage":
> // recurse...
> NodeIterator nit = node.getNodes();
> while (nit.hasNext()) {
> processNode(nsession, nit.nextNode());
> }
> break;
> case JcrConstants.NT_VERSIONHISTORY:
> String vuuid =
> node.getProperty(JcrConstants.JCR_VERSIONABLEUUID).getString();
> if (!nodeExists(nsession, vuuid)) {
> node.remove();
> LOG.info(String.format(">>> Node with UUID %s
> is not stored anymore... its revision history has been deleted...", vuuid));
> }
> break;
> default:
> LOG.warn(String.format("!!! Skipping node %s of
> type %s", npath, ntype));
> break;
> }
> }
> }
>
> private boolean nodeExists(Session nsession, String uuid) {
> try {
> nsession.getNodeByIdentifier(uuid);
> return true;
> } catch (ItemNotFoundException ignoredReturningFalse) {
> return false;
> } catch (RepositoryException ex) {
> LOG.warn(String.format("!!! Got unexpected %s while
> checking if node with UUID %s exists", ex.getMessage(), uuid));
> return false;
> }
> }
> }
>
> When the robot is run, I get this on the log:
>
> 2018-03-16 08:43:06,444INFO
> com.calenco.core.robot.JcrHistoryCleanerTimerTask:42 - class
> com.calenco.core.robot.JcrHistoryCleanerTimerTask running...
>
> 2018-03-16 08:43:06,445INFO
> com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
> /jcr:system/jcr:versionStorage [rep:versionStorage]...
>
> 2018-03-16 08:43:06,448INFO
> com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
> /jcr:system/jcr:versionStorage/78 [rep:versionStorage]...
>
> 2018-03-16 08:43:06,449INFO
> com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
> /jcr:system/jcr:versionStorage/78/11 [rep:versionStorage]...
>
> 2018-03-16 08:43:06,451INFO
> com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
> /jcr:system/jcr:versionStorage/78/11/fe [rep:versionStorage]...
>
> 2018-03-16 08:43:06,454INFO
> com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
> /jcr:system/jcr:versionStorage/78/11/fe/7811fe34-0aae-432a-9015-2626ee77bced
> [nt:versionHistory]...
>
> 2018-03-16 08:43:06,457INFO
> com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
> /jcr:system/jcr:versionStorage/78/da [rep:versionStorage]...
>
> 2018-03-16 08:43:06,459INFO
> com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
> /jcr:system/jcr:versionStorage/78/da/be [rep:versionStorage]...
>
> 2018-03-16 08:43:06,460INFO
> com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
> /jcr:system/jcr:versionStorage/78/da/be/78dabe1c-ead8-414d-8455-4ece9412c1a6
> [nt:versionHistory]...
>
>
> [LOTS_OF_RECURSIVE_TRAVERSAL_RELATED_STUFF_REMOVED_FROM_HERE]
>
>
> 2018-03-16 08:43:06,737INFO
> com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
> /jcr:system/jcr:versionStorage/e9 [rep:versionStorage]...
>
> 2018-03-16 08:43:06,739INFO
> com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
> /jcr:system/jcr:versionStorage/e9/07 [rep:versionStorage]...
>
> 2018-03-16 08:43:06,739INFO
> com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
> /jcr:system/jcr:versionStorage/e9/07/ed [rep:versionStorage]...
>
> 2018-03-16 08:43:06,740INFO
> com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
> /jcr:system/jcr:versionStorage/e9/07/ed/e907edad-ad75-4572-ba44-56a9f5741fb9
> [nt:versionHistory]...
>
> 2018-03-16 08:43:06,743WARN
> com.calenco.core.robot.JcrHistoryCleanerTimerTask:51 - Got
> javax.jcr.nodetype.ConstraintViolationException: Unable to perform
> operation. Node is protected. while trying to clean revision history tree
>
> javax.jcr.nodetype.ConstraintViolationException: Unable to perform
> operation. Node is protected.
>
> at
> org.apache.jackrabbit.core.ItemValidator.checkCondition(ItemValidator.java:276)
>
> at
> org.apache.jackrabbit.core.ItemValidator.checkRemove(ItemValidator.java:254)
>
> at
> org.apache.jackrabbit.core.ItemRemoveOperation.perform(ItemRemoveOperation.java:63)
>
> at
> org.apache.jackrabbit.core.session.SessionState.perform(SessionState.java:200)
>
> at org.apache.jackrabbit.core.ItemImpl.perform(ItemImpl.java:91)
>
> at org.apache.jackrabbit.core.ItemImpl.remove(ItemImpl.java:322)
>
> at
> com.calenco.core.robot.JcrHistoryCleanerTimerTask.processNode(JcrHistoryCleanerTimerTask.java:75)
>
> at
> com.calenco.core.robot.JcrHistoryCleanerTimerTask.processNode(JcrHistoryCleanerTimerTask.java:69)
>
> at
> com.calenco.core.robot.JcrHistoryCleanerTimerTask.processNode(JcrHistoryCleanerTimerTask.java:69)
>
> at
> com.calenco.core.robot.JcrHistoryCleanerTimerTask.processNode(JcrHistoryCleanerTimerTask.java:69)
>
> at
> com.calenco.core.robot.JcrHistoryCleanerTimerTask.processNode(JcrHistoryCleanerTimerTask.java:69)
>
> at
> com.calenco.core.robot.JcrHistoryCleanerTimerTask.run(JcrHistoryCleanerTimerTask.java:46)
>
> at java.util.TimerThread.mainLoop(Timer.java:555)
>
> at java.util.TimerThread.run(Timer.java:505)
>
>
> I've removed lots of log entries for nodes that don't need processing so
> you don't have to scroll down that much to get to the interesting part.
> The removed entries are basically the recursive traversal of the
> revision history tree nodes. Version storage for node with UUID
> e907edad-ad75-4572-ba44-56a9f5741fb9 is an orphan, there's no such node
> stored anymore.
>
> So, it seems that (soundly, but won't let me remove what I need) the
> tree under /jcr:system/jcr:versionStorage is protected in a way that
> some of the "regular" node manipulation API (node.remove()) cannot be
> used on those nodes.
> ...
What type of node are you trying to delete? The version history node?
Best regards, Julian
Re: Version history of deleted nodes and garbage collection
Posted by Fabián Mandelbaum <fm...@gmail.com>.
Hello, the saga continues... I've wrote a small robot to traverse the
versions storage and find the unreferenced version histories (actually, it
finds all nt:versionHistory nodes whose jcr:versionableUuid point to a
non-stored-anymore node).
Using an old-but-still-working-at-least-with-JR-2.6 Java GUI tool to browse
and manipulate the repository, I've deleted some versionable nodes to
create the orphans.
The full code (minus the copyright) for the robot is this:
package com.calenco.core.robot;
import com.calenco.storage.PooledRepoManager;
import java.util.TimerTask;
import javax.jcr.ItemNotFoundException;
import javax.jcr.Node;
import javax.jcr.NodeIterator;
import javax.jcr.RepositoryException;
import javax.jcr.Session;
import org.apache.jackrabbit.JcrConstants;
import org.apache.log4j.Logger;
/**
*
* Task that will periodically check orphaned revision history nodes
* (nt:versionHistory) on the jcr:versionStorage tree and delete the orphans
* found. Ideally, it should not find any orphan... but V3.1 and previous
* Calenco versions left some orphaned revision history nodes on certain
Eraser
* extension operations.
*
* @author fabman
*/
public class JcrHistoryCleanerTimerTask extends TimerTask {
public static final Logger LOG =
Logger.getLogger(JcrHistoryCleanerTimerTask.class);
private static final PooledRepoManager REPOMGR =
PooledRepoManager.getInstance();
public JcrHistoryCleanerTimerTask() {
super();
}
@Override
public void run() {
LOG.info(String.format("%s running...", getClass()));
Session nsession = null;
try {
nsession = REPOMGR.getSessionRW();
processNode(nsession,
nsession.getNode("/jcr:system/jcr:versionStorage"));
if (nsession.hasPendingChanges()) {
nsession.save();
}
} catch (RepositoryException ex) {
LOG.warn(String.format("Got %s while trying to clean revision
history tree", ex), ex);
} finally {
REPOMGR.releaseSessionRW(nsession);
}
}
private void processNode(Session nsession, Node node) throws
RepositoryException {
String npath = node.getPath();
String ntype = node.getPrimaryNodeType().getName();
LOG.info(String.format("Processing node %s [%s]...", npath, ntype));
if (null == ntype) {
LOG.warn(String.format("!!! Skipping node %s of type %s",
npath, ntype));
} else {
switch (ntype) {
case "rep:versionStorage":
// recurse...
NodeIterator nit = node.getNodes();
while (nit.hasNext()) {
processNode(nsession, nit.nextNode());
}
break;
case JcrConstants.NT_VERSIONHISTORY:
String vuuid =
node.getProperty(JcrConstants.JCR_VERSIONABLEUUID).getString();
if (!nodeExists(nsession, vuuid)) {
node.remove();
LOG.info(String.format(">>> Node with UUID %s is
not stored anymore... its revision history has been deleted...", vuuid));
}
break;
default:
LOG.warn(String.format("!!! Skipping node %s of type
%s", npath, ntype));
break;
}
}
}
private boolean nodeExists(Session nsession, String uuid) {
try {
nsession.getNodeByIdentifier(uuid);
return true;
} catch (ItemNotFoundException ignoredReturningFalse) {
return false;
} catch (RepositoryException ex) {
LOG.warn(String.format("!!! Got unexpected %s while checking if
node with UUID %s exists", ex.getMessage(), uuid));
return false;
}
}
}
When the robot is run, I get this on the log:
2018-03-16 08:43:06,444 INFO
com.calenco.core.robot.JcrHistoryCleanerTimerTask:42 - class
com.calenco.core.robot.JcrHistoryCleanerTimerTask running...
2018-03-16 08:43:06,445 INFO
com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
/jcr:system/jcr:versionStorage [rep:versionStorage]...
2018-03-16 08:43:06,448 INFO
com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
/jcr:system/jcr:versionStorage/78 [rep:versionStorage]...
2018-03-16 08:43:06,449 INFO
com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
/jcr:system/jcr:versionStorage/78/11 [rep:versionStorage]...
2018-03-16 08:43:06,451 INFO
com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
/jcr:system/jcr:versionStorage/78/11/fe [rep:versionStorage]...
2018-03-16 08:43:06,454 INFO
com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
/jcr:system/jcr:versionStorage/78/11/fe/7811fe34-0aae-432a-9015-2626ee77bced
[nt:versionHistory]...
2018-03-16 08:43:06,457 INFO
com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
/jcr:system/jcr:versionStorage/78/da [rep:versionStorage]...
2018-03-16 08:43:06,459 INFO
com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
/jcr:system/jcr:versionStorage/78/da/be [rep:versionStorage]...
2018-03-16 08:43:06,460 INFO
com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
/jcr:system/jcr:versionStorage/78/da/be/78dabe1c-ead8-414d-8455-4ece9412c1a6
[nt:versionHistory]...
[LOTS_OF_RECURSIVE_TRAVERSAL_RELATED_STUFF_REMOVED_FROM_HERE]
2018-03-16 08:43:06,737 INFO
com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
/jcr:system/jcr:versionStorage/e9 [rep:versionStorage]...
2018-03-16 08:43:06,739 INFO
com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
/jcr:system/jcr:versionStorage/e9/07 [rep:versionStorage]...
2018-03-16 08:43:06,739 INFO
com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
/jcr:system/jcr:versionStorage/e9/07/ed [rep:versionStorage]...
2018-03-16 08:43:06,740 INFO
com.calenco.core.robot.JcrHistoryCleanerTimerTask:60 - Processing node
/jcr:system/jcr:versionStorage/e9/07/ed/e907edad-ad75-4572-ba44-56a9f5741fb9
[nt:versionHistory]...
2018-03-16 08:43:06,743 WARN
com.calenco.core.robot.JcrHistoryCleanerTimerTask:51 - Got
javax.jcr.nodetype.ConstraintViolationException: Unable to perform
operation. Node is protected. while trying to clean revision history tree
javax.jcr.nodetype.ConstraintViolationException: Unable to perform
operation. Node is protected.
at
org.apache.jackrabbit.core.ItemValidator.checkCondition(ItemValidator.java:276)
at
org.apache.jackrabbit.core.ItemValidator.checkRemove(ItemValidator.java:254)
at
org.apache.jackrabbit.core.ItemRemoveOperation.perform(ItemRemoveOperation.java:63)
at
org.apache.jackrabbit.core.session.SessionState.perform(SessionState.java:200)
at org.apache.jackrabbit.core.ItemImpl.perform(ItemImpl.java:91)
at org.apache.jackrabbit.core.ItemImpl.remove(ItemImpl.java:322)
at
com.calenco.core.robot.JcrHistoryCleanerTimerTask.processNode(JcrHistoryCleanerTimerTask.java:75)
at
com.calenco.core.robot.JcrHistoryCleanerTimerTask.processNode(JcrHistoryCleanerTimerTask.java:69)
at
com.calenco.core.robot.JcrHistoryCleanerTimerTask.processNode(JcrHistoryCleanerTimerTask.java:69)
at
com.calenco.core.robot.JcrHistoryCleanerTimerTask.processNode(JcrHistoryCleanerTimerTask.java:69)
at
com.calenco.core.robot.JcrHistoryCleanerTimerTask.processNode(JcrHistoryCleanerTimerTask.java:69)
at
com.calenco.core.robot.JcrHistoryCleanerTimerTask.run(JcrHistoryCleanerTimerTask.java:46)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
I've removed lots of log entries for nodes that don't need processing so
you don't have to scroll down that much to get to the interesting part. The
removed entries are basically the recursive traversal of the revision
history tree nodes. Version storage for node with UUID
e907edad-ad75-4572-ba44-56a9f5741fb9 is an orphan, there's no such node
stored anymore.
So, it seems that (soundly, but won't let me remove what I need) the tree
under /jcr:system/jcr:versionStorage is protected in a way that some of the
"regular" node manipulation API (node.remove()) cannot be used on those
nodes.
Our application has a JCR garbage collector programmed to run. I let it run:
2018-03-16 09:00:00,205 INFO com.calenco.storage.PooledRepoManager:757 -
Starting repository garbage collection...
2018-03-16 09:00:00,456 INFO com.calenco.storage.PooledRepoManager:770 -
Finished repository garbage collection.
Then stopped the application and used the tool to inspect the repository...
I still find the orphan version history nodes, which take up useless disk
space and all their references are actually pointing nowhere (to no stored
node, that's what defines them as orphans in the 1st place...). So the
orphaned version history nodes are not eligible for garbage collection (one
of my original questions, answered now).
Even if I know disk space is cheap these days, I don't want JCR to store
useless garbage for my application's data (our current repository is 60GB
and growing...)
Is there a way to manipulate those nodes under
/jcr:system/jcr:versionStorage with the JCR API so I can remove the orphans?
Thanks once more for your help, and sorry for the long, detailed, message.
Best regards.
On Thu, Mar 15, 2018 at 7:54 AM, Julian Reschke <ju...@gmx.de>
wrote:
> On 2018-03-15 11:47, Fabián Mandelbaum wrote:
>
>> I'll try to provide the details for dependency hell with 2.6 and other
>> frameworks/libs later, I'm a bit overloaded with other things now, sorry.
>>
>
> Thanks.
>
> Nothing happens with the tree, OK, but is it eligible to be
>> garbage-collected or not?
>>
>
> I wouldn't call it "garbage" :-) But yes, it should be possible to delete
> those.
>
> Indeed, if you try to delete a referenced node you get a
>> ReferentialIntegrityException, which is fine. Now, I'll be fine with
>> writing a robot (periodically-run task) to find all such orphans and wipe
>> them. How can I detect those orphans SAFELY (in a way that attempting to
>> delete them will not throw ReferentialIntegrityException, and will not
>> break my content repository in any way). Is there a ready-made utility
>> method in the API or a query, or set of queries, I can run to detect those
>> orphans?
>>
>> Thanks again for your help.
>>
>
> I'm not sure there's a way to write a query for this.
>
> You should be able to traverse the versions storage and find the
> unreferenced version histories. That said, why don't you simply collect the
> paths while deleting the version-controlled nodes?
>
> Best regards, Julian
>
--
Fabián Mandelbaum
IS Engineer
Re: Version history of deleted nodes and garbage collection
Posted by Julian Reschke <ju...@gmx.de>.
On 2018-03-15 11:47, Fabián Mandelbaum wrote:
> I'll try to provide the details for dependency hell with 2.6 and other
> frameworks/libs later, I'm a bit overloaded with other things now, sorry.
Thanks.
> Nothing happens with the tree, OK, but is it eligible to be
> garbage-collected or not?
I wouldn't call it "garbage" :-) But yes, it should be possible to
delete those.
> Indeed, if you try to delete a referenced node you get a
> ReferentialIntegrityException, which is fine. Now, I'll be fine with
> writing a robot (periodically-run task) to find all such orphans and
> wipe them. How can I detect those orphans SAFELY (in a way that
> attempting to delete them will not throw ReferentialIntegrityException,
> and will not break my content repository in any way). Is there a
> ready-made utility method in the API or a query, or set of queries, I
> can run to detect those orphans?
>
> Thanks again for your help.
I'm not sure there's a way to write a query for this.
You should be able to traverse the versions storage and find the
unreferenced version histories. That said, why don't you simply collect
the paths while deleting the version-controlled nodes?
Best regards, Julian
Re: Version history of deleted nodes and garbage collection
Posted by Fabián Mandelbaum <fm...@gmail.com>.
I'll try to provide the details for dependency hell with 2.6 and other
frameworks/libs later, I'm a bit overloaded with other things now, sorry.
Nothing happens with the tree, OK, but is it eligible to be
garbage-collected or not?
Indeed, if you try to delete a referenced node you get a
ReferentialIntegrityException, which is fine. Now, I'll be fine with
writing a robot (periodically-run task) to find all such orphans and wipe
them. How can I detect those orphans SAFELY (in a way that attempting to
delete them will not throw ReferentialIntegrityException, and will not
break my content repository in any way). Is there a ready-made utility
method in the API or a query, or set of queries, I can run to detect those
orphans?
Thanks again for your help.
Best regards, Fabián
On Thu, Mar 15, 2018 at 7:24 AM, Julian Reschke <ju...@gmx.de>
wrote:
> On 2018-03-15 10:47, Fabián Mandelbaum wrote:
>
>> Hello,
>>
>> my application based on JackRabbit 2.x (albeit an old .x, IIRC, .6, still
>> cannot upgrade to latest because it breaks my build... lots of
>> dependencies
>>
>
> Details appreciated. We'll abandon 2.6 this year.
>
> from other libs/frameworks I use) uses version history tracking of items
>> stored under certain paths.
>>
> >
>
>> One of the features of the application is to be able to erase the items
>> (wipe
>> them, and all related nodes and stored information forever) in question.
>>
>> When deleting those items, what happens with their version history tree?
>> Is
>>
>
> By default, nothing. (I think)
>
> the version history tree eligible for garbage collection (ran by a "robot"
>> in my app once a day)? Or should I also delete the version history tree
>> "by
>> hand" (which poses lots of problems with session state... I have to do
>> session.save() 'in the middle' of the deletion process)?
>>
>> Thanks in advance for your answers.
>>
>
> I think you can delete the version history if and only iff there is no
> version controlled node referring to it anymore.
>
> Best regards, Julian
>
--
Fabián Mandelbaum
IS Engineer
Re: Version history of deleted nodes and garbage collection
Posted by Julian Reschke <ju...@gmx.de>.
On 2018-03-15 10:47, Fabián Mandelbaum wrote:
> Hello,
>
> my application based on JackRabbit 2.x (albeit an old .x, IIRC, .6, still
> cannot upgrade to latest because it breaks my build... lots of dependencies
Details appreciated. We'll abandon 2.6 this year.
> from other libs/frameworks I use) uses version history tracking of items
> stored under certain paths.
>
> One of the features of the application is to be able to erase the items (wipe
> them, and all related nodes and stored information forever) in question.
>
> When deleting those items, what happens with their version history tree? Is
By default, nothing. (I think)
> the version history tree eligible for garbage collection (ran by a "robot"
> in my app once a day)? Or should I also delete the version history tree "by
> hand" (which poses lots of problems with session state... I have to do
> session.save() 'in the middle' of the deletion process)?
>
> Thanks in advance for your answers.
I think you can delete the version history if and only iff there is no
version controlled node referring to it anymore.
Best regards, Julian