You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Tanya Levshina <ta...@volvovski.com> on 2007/06/22 01:34:44 UTC
Problem using RAMDirectory as a buffer
Hi,
I am trying to use RAMDirectory as a buffer and am having some problems. I
create indexes using FSDirectory directly and index directory contains the
following files:
bash-3.00$ ls ~/index/
_0.cfs segments_3 segments.gen
When I am trying to use RAMDirectory as a buffer and then add indexes to a
FSDirectory the directory looks like:
-bash-3.00$ ls ~/index/
segments_1 segments.gen
and search is failing.
This is a code I use to do it:
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.FSDirectory;
import org.apache.lucene.store.RAMDirectory;
import java.io.FileReader;
class Test {
public static void main(String[] args) throws Exception{
FSDirectory fsDir= FSDirectory.getDirectory(args[0]);
RAMDirectory ramDir=new RAMDirectory();
IndexWriter fsWriter=new IndexWriter(fsDir,new
StandardAnalyzer(),true);
IndexWriter ramWriter=new IndexWriter(ramDir,new
StandardAnalyzer(),true);
Document doc=new Document();
doc.add(new Field("test","I am testing RAM directory",
Field.Store.YES, Field.Index.UN_TOKENIZED));
doc.add(new Field("contents", new
FileReader(args[1]),Field.TermVector.WITH_POSITIONS_OFFSETS ) );
ramWriter.addDocument(doc);
fsWriter.addIndexes(new Directory[] {ramDir,});
ramWriter.close();
fsWriter.optimize();
fsWriter.close();
}
}
To run it just create index directory and do:
java Test <index_dir_name> <some_file_name>
Any idea?
Thanks a lot,
Tanya
RE: Problem using RAMDirectory as a buffer
Posted by Tanya Levshina <ta...@volvovski.com>.
Hi,
This example is taken almost exactly from "Lucene in Action" page #51.
I would like to use RAMDirectory to add files and then flush it to disk.
I've tried the most trivial example and it doesnt work.
So obviously I am doing something wrong.
Tanya
-----Original Message-----
From: Daniel Noll [mailto:daniel@nuix.com]
Sent: Thursday, June 21, 2007 6:39 PM
To: java-user@lucene.apache.org
Subject: Re: Problem using RAMDirectory as a buffer
On Friday 22 June 2007 09:34:44 Tanya Levshina wrote:
> ramWriter.addDocument(doc);
>
> fsWriter.addIndexes(new Directory[] {ramDir,});
As IndexWriter already does this internally, I'm not exactly sure why you're
trying to implement it again on the outside.
Daniel
--
Daniel Noll
Nuix Pty Ltd
Suite 79, 89 Jones St, Ultimo NSW 2007, Australia Ph: +61 2 9280 0699
Web: http://nuix.com/ Fax: +61 2 9212 6902
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Problem using RAMDirectory as a buffer
Posted by Tanya Levshina <ta...@volvovski.com>.
Hi,
I confirm that it is working correctly in lucene 2.2.0 and not working in
2.1.0.
Thanks a lot,
Tanya
-----Original Message-----
From: Michael Busch [mailto:buschmic@gmail.com]
Sent: Thursday, June 21, 2007 7:25 PM
To: java-user@lucene.apache.org
Subject: Re: Problem using RAMDirectory as a buffer
Tanya Levshina wrote:
> Nope, doesn't work.
> I've tried:
> ....
> ramWriter.addDocument(doc);
> ramWriter.flush();
> ramWriter.close();
> fsWriter.addIndexes(new Directory[] {ramDir,});
> ....
>
>
> Any other suggestions?
>
Are you sure? That's strange, I just took your code and tried it out
myself. With my suggested change it works fine for me now. I'm using
Lucene 2.2 for this test.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Problem using RAMDirectory as a buffer
Posted by Tanya Levshina <ta...@volvovski.com>.
Hi,
I am using lucene 2.1.0. I'll try to download 2.2 and test it.
Thanks,
Tanya
-----Original Message-----
From: Michael Busch [mailto:buschmic@gmail.com]
Sent: Thursday, June 21, 2007 7:25 PM
To: java-user@lucene.apache.org
Subject: Re: Problem using RAMDirectory as a buffer
Tanya Levshina wrote:
> Nope, doesn't work.
> I've tried:
> ....
> ramWriter.addDocument(doc);
> ramWriter.flush();
> ramWriter.close();
> fsWriter.addIndexes(new Directory[] {ramDir,});
> ....
>
>
> Any other suggestions?
>
Are you sure? That's strange, I just took your code and tried it out
myself. With my suggested change it works fine for me now. I'm using
Lucene 2.2 for this test.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Problem using RAMDirectory as a buffer
Posted by Michael Busch <bu...@gmail.com>.
Tanya Levshina wrote:
> Nope, doesn't work.
> I've tried:
> ....
> ramWriter.addDocument(doc);
> ramWriter.flush();
> ramWriter.close();
> fsWriter.addIndexes(new Directory[] {ramDir,});
> ....
>
>
> Any other suggestions?
>
Are you sure? That's strange, I just took your code and tried it out
myself. With my suggested change it works fine for me now. I'm using
Lucene 2.2 for this test.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Problem using RAMDirectory as a buffer
Posted by Tanya Levshina <ta...@volvovski.com>.
Nope, doesn't work.
I've tried:
....
ramWriter.addDocument(doc);
ramWriter.flush();
ramWriter.close();
fsWriter.addIndexes(new Directory[] {ramDir,});
....
Any other suggestions?
Thanks,
Tanya
PS As I said this is just a trivial test I created because it didn't work
for more complicated problem
-----Original Message-----
From: Michael Busch [mailto:buschmic@gmail.com]
Sent: Thursday, June 21, 2007 6:57 PM
To: java-user@lucene.apache.org
Subject: Re: Problem using RAMDirectory as a buffer
Daniel Noll wrote:
> On Friday 22 June 2007 09:34:44 Tanya Levshina wrote:
>> ramWriter.addDocument(doc);
>>
>> fsWriter.addIndexes(new Directory[] {ramDir,});
>
> As IndexWriter already does this internally, I'm not exactly sure why
you're
> trying to implement it again on the outside.
>
> Daniel
>
You should close or flush the ramWriter before you add the ramDir to the
fsWriter, otherwise the docs won't get flushed to the ramDir.
But I share Daniel's opinion. IndexWriter has it's own buffer internally.
- Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Problem using RAMDirectory as a buffer
Posted by Michael Busch <bu...@gmail.com>.
Daniel Noll wrote:
> On Friday 22 June 2007 09:34:44 Tanya Levshina wrote:
>> ramWriter.addDocument(doc);
>>
>> fsWriter.addIndexes(new Directory[] {ramDir,});
>
> As IndexWriter already does this internally, I'm not exactly sure why you're
> trying to implement it again on the outside.
>
> Daniel
>
You should close or flush the ramWriter before you add the ramDir to the
fsWriter, otherwise the docs won't get flushed to the ramDir.
But I share Daniel's opinion. IndexWriter has it's own buffer internally.
- Michael
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Problem using RAMDirectory as a buffer
Posted by Daniel Noll <da...@nuix.com>.
On Friday 22 June 2007 09:34:44 Tanya Levshina wrote:
> ramWriter.addDocument(doc);
>
> fsWriter.addIndexes(new Directory[] {ramDir,});
As IndexWriter already does this internally, I'm not exactly sure why you're
trying to implement it again on the outside.
Daniel
--
Daniel Noll
Nuix Pty Ltd
Suite 79, 89 Jones St, Ultimo NSW 2007, Australia Ph: +61 2 9280 0699
Web: http://nuix.com/ Fax: +61 2 9212 6902
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org