Coping documents across indexes (Was: How to extract matching terms for a document given a query)

[Giulio Cesare Solaroli <gi...@gmail.com>]

On Wed, 16 Jun 2004 23:50:50 +0200, Giulio Cesare Solaroli
<gi...@gmail.com> wrote:
> 
> On Wed, 16 Jun 2004 22:31:07 +0100, markharw00d@yahoo.co.uk
> <ma...@yahoo.co.uk> wrote:
> >
> > Yes, highlighting multi-term queries does require a query.rewrite() call to expand those terms before
> > calling the highlighter.
> > BUT, you could load the results documents into a temporary RAMDirectory and expand the query by rewriting it
> > against THAT instead of the original index - it would still produce the term expansions you need.
> 
> That's a nice idea. The rewritten query would be much smaller when
> expanded against an index containing only the selected documents.

I have thought more about this option, and I have tried to implement
the following policy:
- find the matching documents (Hits);
- create an IndexWriter associated to a RamDirectory;
- for each document in the request range
   * process the document;
   * insert the document into the RamDirectory index
         ramDiskWriter.addDocument(hits.doc(i));
- when all the required documents have been processed (and inserted
into the new ramDirectory index) rewrite the original query against
this new index.

But this looks too nice to be true.

Indeed, the rewritten query (against the the index) is empty. :-[

Does anybody know if it is possible to copy documents across indexes
for this purpose?

Giulio Cesare Solaroli

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org

Re: Coping documents across indexes (Was: How to extract matching terms for a document given a query)

[Giulio Cesare Solaroli <gi...@gmail.com>]

Replying to my own post ...

> > That's a nice idea. The rewritten query would be much smaller when
> > expanded against an index containing only the selected documents.
> 
> I have thought more about this option, and I have tried to implement
> the following policy:
> - find the matching documents (Hits);
> - create an IndexWriter associated to a RamDirectory;
> - for each document in the request range
>    * process the document;
>    * insert the document into the RamDirectory index
>          ramDiskWriter.addDocument(hits.doc(i));
> - when all the required documents have been processed (and inserted
> into the new ramDirectory index) rewrite the original query against
> this new index.
> 
> But this looks too nice to be true.
> 
> Indeed, the rewritten query (against the the index) is empty. :-[
> 
> Does anybody know if it is possible to copy documents across indexes
> for this purpose?

The documents are copied, but the new document contains only the
fields stored in the first index. :-[

Is it possible to copy document terms across  indexes, even if the
fields are not stored?

Thanks for your attention,

Giulio Cesare Solaroli

---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org