An alternative for TDBMaker.clearDatasetCache() ?

[Paolo Castagna <ca...@googlemail.com>]

Hi,
at one point, if TDBMaker we had a method called clearDatasetCache().

I have been using that method in a @Before setup method of a test [1]:

    @Before public void setup() {
        if ( FileOps.exists(output) ) {
            FileOps.clearDirectory(output) ;
        } else {
            FileOps.ensureDir(output);
        }
        // I don't understand why this is necessary... :-/
        TDBMaker.clearDatasetCache();
    }

Notice the silly comment.

The test loads some data into a TDB store on disk:

        Location location = new Location(output);
        DatasetGraphTDB dsgDisk =
(DatasetGraphTDB)TDBFactory.createDatasetGraph(location);
        TDBLoader.load(dsgDisk, urls);

It loads the same data in memory:

        DatasetGraphTDB dsgMem = (DatasetGraphTDB)TDBFactory.createDatasetGraph();
        TDBLoader.load(dsgMem, urls);

It deletes and regenerate the node2id.idn and node2id.dat files on disk:

        NodeTableRewriter.fixNodeTable2(location, log, null);

Finally, it asserts the memory and disk TDB datasets are isomorphic:

        assertTrue ( tdbloader4.dump(dsgMem, dsgDisk), tdbloader4.isomorphic (
dsgMem, dsgDisk ) );

Everything is fine, datasets are isomorphic.

Now, I am trying to update to TDB 0.9.0-incubating in staging at the moment.

The method TDBMaker.clearDatasetCache() is not there anymore.

I tried to replace it:
-        TDBMaker.clearDatasetCache();
+        ((CachingTDBMaker)TDBMaker.cachedFactory).flush();

But, it did not work, I now have 3 failures.

So, I tried to use TDBMaker.uncachedFactory.createDatasetGraph(location):

-        DatasetGraphTDB dsgDisk =
(DatasetGraphTDB)TDBFactory.createDatasetGraph(location);
+        DatasetGraphTDB dsgDisk =
TDBMaker.uncachedFactory.createDatasetGraph(location);
         TDBLoader.load(dsgDisk, urls);
-
-        DatasetGraphTDB dsgMem = (DatasetGraphTDB)TDBFactory.createDatasetGraph();
-        TDBLoader.load(dsgMem, urls);
+
+        DatasetGraphTransaction dsgMem =
(DatasetGraphTransaction)TDBFactory.createDatasetGraph();
+        TDBLoader.load(dsgMem.getBaseDatasetGraph(), urls);

This worked.

What's the best way to interact with TDBMaker and clean/reset/flush caches?

Thanks,
Paolo

 [1]
https://github.com/castagna/tdbloader4/blob/master/src/test/java/org/apache/jena/tdbloader4/TestNodeTableRewriter.java

Re: An alternative for TDBMaker.clearDatasetCache() ?

[Andy Seaborne <an...@apache.org>]

On 07/03/12 12:58, Paolo Castagna wrote:
> Hi,
> at one point, if TDBMaker we had a method called clearDatasetCache().

 > TDB 0.9.0-incubating

Be careful - this is no longer the current build of trunk which is 
0.9.1-incubating

> What's the best way to interact with TDBMaker and clean/reset/flush caches?

The best way is not to :-)  TDBMaker isn't used any more by the API.

Warning - TDBMaker will be deleted unless there is a strong reason not 
to, because it is in parallel and separate from the StoreConnection 
management of transactional datasets.

To get the raw storage DatasetGraph,

     SystemTDB.getBaseDatasetGraphTDB(DatasetGraph)


If you are using TDBMaker directly:

    TDBMaker.reset()

exists.

To clear the cache of managed datasets things in the new transaction world:

StoreConnection.reset()

This does not abort active transactions (you can't).

There is

TDBMakerTxn.reset()

but since TDBMakerTxn is just a hop to move implementation foo out of 
(public) TDBFactory,

	Andy


>
> Thanks,
> Paolo
>
>   [1]
> https://github.com/castagna/tdbloader4/blob/master/src/test/java/org/apache/jena/tdbloader4/TestNodeTableRewriter.java