Fwd: Speed up initial Bundle load > 10k from database

[Alexej Sailer <al...@gmail.com>]

---------- Forwarded message ----------
From: Alexej Sailer <al...@gmail.com>
Date: 2016-08-02 10:13 GMT+02:00
Subject: Speed up initial Bundle load > 10k from database
To: users@jackrabbit.apache.org


I'm using jackrabbit and from time to time it happens, that I have more
than 10k Childnodes. So I noticed that in this case jackrabbit slows down
loading them initially into the bundle cache. After debugging I found the
method protected NodePropBundle loadBundle(NodeId id) in
OraclePersistenceManager and there, every node bundle is loaded one by one,
which is extremely slow after 2k bundles.

So I wrote a simple workaround: I start a bunch off threads at exactly this
method and load all bundles from the database at once for 60 000 it takes
me 5 sec I think, and I cache them exactly in this place, so that If I
start loading > 10k child nodes one by one, they are loaded from this
cache. And that speeds up the load 10x.

This is simply a bundle cache but, which is filled up extremely fast.

Now, the cache invalidation is done by using protected and public methods
from PersistenceManager.

It works fine. But I am not sure whether this is consistent in Apache
JackRabbit implementation.

Who can answer this questions? I will provide the code as well, if there
are interests to that.

BR,

Alexej Sailer






protected synchronized void storeBundle(NodePropBundle bundle) throws
ItemStateException
{

AsyncBundleReader.getNodePropBundleCache().getCache().invalidate(bundle.getId().toString());
        super.storeBundle(bundle);
}

@Override
    protected synchronized void destroyBundle(NodePropBundle bundle) throws
ItemStateException
    {

AsyncBundleReader.getNodePropBundleCache().getCache().invalidate(bundle.getId().toString());
        super.destroyBundle(bundle);
    }


    @Override
    protected void evictBundle(NodeId id)
    {

AsyncBundleReader.getNodePropBundleCache().getCache().invalidate(id.toString());
        super.evictBundle(id);
    }


 @Override
    public synchronized void onExternalUpdate(ChangeLog changes)
    {
        for (ItemState state : changes.modifiedStates())
        {

AsyncBundleReader.getNodePropBundleCache().getCache().invalidate(getBundleId(state).toString());
        }
        for (ItemState state : changes.deletedStates())
        {

AsyncBundleReader.getNodePropBundleCache().getCache().invalidate(getBundleId(state).toString());
        }
        for (ItemState state : changes.addedStates())
        {

AsyncBundleReader.getNodePropBundleCache().getCache().invalidate(getBundleId(state).toString());
        }
        super.onExternalUpdate(changes);
    }