highlighting and external storage

[Thomas Koch <th...@koch.ro>]

Hi,

I'm working on a news crawler with continuous indexing. Thus indexes are 
merged frequently and older documents aren't as important as recent ones.

Therefor I'd like to store the fulltext of documents in an external storage 
(HBase?) so that merging of indexes isn't as IO intensive. This would give me 
the additional benefit, that I could selectively delete the fulltext of older 
articles when running out of disc space while keeping the url of the document 
in the index.

Do you know, whether sth. like this would be possible?

Best regards,

Thomas Koch, http://www.koch.ro

Re: highlighting and external storage

[Erik Hatcher <er...@gmail.com>]

Thomas - this is a common need that deserves some implementation.  I  
have a personal interest in seeing this implemented and will do so  
myself eventually if no one beats me to it.

There's a Solr JIRA issue to track this:  https://issues.apache.org/jira/browse/SOLR-1397

	Erik

On Dec 22, 2009, at 12:06 PM, Thomas Koch wrote:

> Hi,
>
> I'm working on a news crawler with continuous indexing. Thus indexes  
> are
> merged frequently and older documents aren't as important as recent  
> ones.
>
> Therefor I'd like to store the fulltext of documents in an external  
> storage
> (HBase?) so that merging of indexes isn't as IO intensive. This  
> would give me
> the additional benefit, that I could selectively delete the fulltext  
> of older
> articles when running out of disc space while keeping the url of the  
> document
> in the index.
>
> Do you know, whether sth. like this would be possible?
>
> Best regards,
>
> Thomas Koch, http://www.koch.ro