You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@jena.apache.org by Phillip Rhodes <mo...@gmail.com> on 2014/10/10 02:50:58 UTC
jena-text and persistent lucene index
Jena crew:
I have Jena-Text pretty much working for what I want to do, but with
one caveat... it only seems to work with the Lucene RamDirectory. If
I use a persistent dir, I don't find any documents being written to
the Lucene index. Any help or suggestions are greatly appreciated.
Code sample follows:
---CODE---
package org.fogbeam.example.jenatext
import org.apache.jena.query.text.EntityDefinition
import org.apache.jena.query.text.TextDatasetFactory
import org.apache.jena.query.text.TextQuery
import org.apache.lucene.store.Directory
import org.apache.lucene.store.FSDirectory
import com.hp.hpl.jena.query.Dataset
import com.hp.hpl.jena.query.ReadWrite
import com.hp.hpl.jena.rdf.model.Model
import com.hp.hpl.jena.rdf.model.Resource
import com.hp.hpl.jena.rdf.model.Statement
import com.hp.hpl.jena.tdb.TDBFactory
import com.hp.hpl.jena.vocabulary.RDFS
class JenaTextMain6Write
{
static main(args)
{
TextQuery.init();
EntityDefinition entDef = new EntityDefinition("uri", "text",
RDFS.label.asNode()) ;
// Lucene, in memory.
Directory dir = FSDirectory.open( new File( "jenastore/index" ) );
// Join together into a dataset
Dataset tdbDataset = TDBFactory.createDataset("jenastore/triples");
Dataset ds = TextDatasetFactory.createLucene(tdbDataset, dir, entDef);
try
{
ds.begin(ReadWrite.WRITE);
Model m = ds.getDefaultModel();
Resource rSubject = m.createResource(
"http://ontology.fogbeam.com/example/TestResource1" );
Resource rSubject2 = m.createResource(
"http://ontology.fogbeam.com/example/TestResource2" );
Statement s = m.createStatement(rSubject, RDFS.label,
"This is a Test Resource" );
m.add( s );
Statement s2 = m.createStatement(rSubject2, RDFS.label,
"Bratwurst Test" );
m.add( s2 );
ds.commit();
dir.close();
}
catch( Exception e )
{
e.printStackTrace();
ds.abort();
}
finally
{
if( ds != null )
{
ds.end();
}
}
println "done";
}
}
--- END CODE---
Phil
---
This message optimized for indexing by NSA PRISM
Re: jena-text and persistent lucene index
Posted by Andy Seaborne <an...@apache.org>.
On 10/10/14 17:16, Phillip Rhodes wrote:
> On Fri, Oct 10, 2014 at 4:01 AM, Andy Seaborne <an...@apache.org> wrote:
>>
>> How do you know it does not work? The code (groovy?) does not read the
>> index.
>
> Hey Andy, I had written a separate reader program to test using the
> index for queries. I did
> it that way to make sure I was seeing results from using the index,
> and not accidentally getting
> something that was cached in RAM or something.
>
> Anyway, as it turns out, I managed to solve the problem. As best as I
> can tell, the issue
> was this, or something close: The first time(s) I ran this program,
> I didn't yet have a
> dir.close() line in there to close the Lucene Directory. As a result,
> I was getting
> an incomplete Lucene index. Later, after I fixed that, I kept running
> the program
> but since I was creating the exact same triples (and not deleting the
> TDB dir between
> runs) Jena-Text didn't reindex those triples since they already existed.
>
> Perhaps you (or someone) can confirm if this is the way Jena Text would behave
> in that regard?
Ideally, jena-text would flush the changes to the index on a commit but
at the same time the Directory resource is created outside the jena-text
library so the usual pattern is that the creator is responsible for it's
lifecycle. For example, you can use try-with-resources on it.
If you create the TextIndex in the app (it's hidden in
TextDatasetFactory.createLucene in your code) you can call
.startIndexing/.finishIndexing which would also flush changes,
Andy
>
> Anyway, once I deleted my TDB store and started from scratch,
> everything worked exactly
> as expected.
>
>
> Thanks!
>
>
> Phil
>
Re: jena-text and persistent lucene index
Posted by Phillip Rhodes <mo...@gmail.com>.
On Fri, Oct 10, 2014 at 4:01 AM, Andy Seaborne <an...@apache.org> wrote:
>
> How do you know it does not work? The code (groovy?) does not read the
> index.
Hey Andy, I had written a separate reader program to test using the
index for queries. I did
it that way to make sure I was seeing results from using the index,
and not accidentally getting
something that was cached in RAM or something.
Anyway, as it turns out, I managed to solve the problem. As best as I
can tell, the issue
was this, or something close: The first time(s) I ran this program,
I didn't yet have a
dir.close() line in there to close the Lucene Directory. As a result,
I was getting
an incomplete Lucene index. Later, after I fixed that, I kept running
the program
but since I was creating the exact same triples (and not deleting the
TDB dir between
runs) Jena-Text didn't reindex those triples since they already existed.
Perhaps you (or someone) can confirm if this is the way Jena Text would behave
in that regard?
Anyway, once I deleted my TDB store and started from scratch,
everything worked exactly
as expected.
Thanks!
Phil
Re: jena-text and persistent lucene index
Posted by Andy Seaborne <an...@apache.org>.
Phil,
How do you know it does not work? The code (groovy?) does not read the
index. Could you provide a complete example (I don't have groovy
installed ATM).
(which version are you using?)
Andy
On 10/10/14 01:50, Phillip Rhodes wrote:
> Jena crew:
>
> I have Jena-Text pretty much working for what I want to do, but with
> one caveat... it only seems to work with the Lucene RamDirectory. If
> I use a persistent dir, I don't find any documents being written to
> the Lucene index. Any help or suggestions are greatly appreciated.
> Code sample follows:
>
>
> ---CODE---
>
>
> package org.fogbeam.example.jenatext
>
> import org.apache.jena.query.text.EntityDefinition
> import org.apache.jena.query.text.TextDatasetFactory
> import org.apache.jena.query.text.TextQuery
> import org.apache.lucene.store.Directory
> import org.apache.lucene.store.FSDirectory
>
> import com.hp.hpl.jena.query.Dataset
> import com.hp.hpl.jena.query.ReadWrite
> import com.hp.hpl.jena.rdf.model.Model
> import com.hp.hpl.jena.rdf.model.Resource
> import com.hp.hpl.jena.rdf.model.Statement
> import com.hp.hpl.jena.tdb.TDBFactory
> import com.hp.hpl.jena.vocabulary.RDFS
>
> class JenaTextMain6Write
> {
>
> static main(args)
> {
>
> TextQuery.init();
>
> EntityDefinition entDef = new EntityDefinition("uri", "text",
> RDFS.label.asNode()) ;
>
> // Lucene, in memory.
> Directory dir = FSDirectory.open( new File( "jenastore/index" ) );
>
> // Join together into a dataset
> Dataset tdbDataset = TDBFactory.createDataset("jenastore/triples");
> Dataset ds = TextDatasetFactory.createLucene(tdbDataset, dir, entDef);
>
>
> try
> {
>
> ds.begin(ReadWrite.WRITE);
>
> Model m = ds.getDefaultModel();
>
> Resource rSubject = m.createResource(
> "http://ontology.fogbeam.com/example/TestResource1" );
> Resource rSubject2 = m.createResource(
> "http://ontology.fogbeam.com/example/TestResource2" );
>
>
> Statement s = m.createStatement(rSubject, RDFS.label,
> "This is a Test Resource" );
>
> m.add( s );
>
> Statement s2 = m.createStatement(rSubject2, RDFS.label,
> "Bratwurst Test" );
>
> m.add( s2 );
>
> ds.commit();
>
> dir.close();
>
> }
> catch( Exception e )
> {
> e.printStackTrace();
> ds.abort();
> }
> finally
> {
> if( ds != null )
> {
> ds.end();
> }
> }
>
> println "done";
> }
>
> }
>
> --- END CODE---
>
>
>
> Phil
> ---
> This message optimized for indexing by NSA PRISM
>