inserting data into triplestore with less main memory foot print

[Ganesh kumar <ga...@gmail.com>]

Hi All,

I am trying to insert triples into Jena TDB (triples are added into simple
models and then inserted).
The models are built at runtime for a particular group of triples, and then
it needs to be persisted into Jena TDB.


Pseudocode:

Resource node = getRdfModel().createResource(nodeStr);

for (Entry<String, Object> entry : attributes.entrySet()) {
String attributeName = entry.getKey();

attributeName = processAttributes(attributeName);
propertyStr = buildURI(attributeName, source, elementType.property,
nodeStr);
String val = processValue(entry);

node.addProperty(getRdfModel().createProperty(propertyStr),
getRdfModel().createLiteral(val));
}
 dataset.begin(ReadWrite.WRITE);
try {

// TODO: This insert is not really effecient. The getnamed model loads the
data from the TDB into memory which is making the insert slow. Is there a
effecient way of inserting the data to TDB like a cursor with less memory
usage ?
                       dataset.getNamedModel(source).add(getRdfModel());
dataset.commit();
//After commit the getRDFModel() will be reseted.

} finally {
dataset.end();
}

Problem:

When I call the getNamedmodel function, it loads the data from TDB which is
quite large. Is there a way to insert the data without loading the data
into main memory ?

or any other effecient way of inserting data using Jena api?
This is a data migration study project, so more data will keep flowing in.


Thanks in advance for your help & guidance.

Thanks
Ganesh