indexing frequently-changing fields

[didier deshommes <df...@gmail.com>]

I am using Solr to index data in a SQL database.  Most of the data
doesn't change after initial commit, except for a single boolean field
that indicates whether an item is flagged as 'needing attention'.  So
I have a need_attention field in the database that I update whenever a
user marks an item as needing attention in my UI.  The problem I have
is that I want to offer the ability to include need_attention in my
user's queries, but do not want to incur the expense of having to
reindex whenever this flag changes on an individual document.

I have thought about different solutions to this problem, including
using multi-core and having a smaller core for recently-marked items
that I am willing to do 'near-real-time' commits on.  Are there are
any common solutions to this problem, which I have to imagine is
common in this community?

Re: indexing frequently-changing fields

[Yonik Seeley <yo...@lucidimagination.com>]

It's a bit round-about but you might be able to use ExternalFileField
http://lucene.apache.org/solr/api/org/apache/solr/schema/ExternalFileField.html

The fieldType definition would look like
    <fieldType name="file" keyField="id" defVal="1" stored="false"
indexed="false" class="solr.ExternalFileField" valType="float"/>

Then you can use frange to include/exclude certain values:

http://www.lucidimagination.com/blog/tag/frange/

-Yonik
http://www.lucidimagination.com


On Thu, Oct 8, 2009 at 4:59 PM, didier deshommes <df...@gmail.com> wrote:
> I am using Solr to index data in a SQL database.  Most of the data
> doesn't change after initial commit, except for a single boolean field
> that indicates whether an item is flagged as 'needing attention'.  So
> I have a need_attention field in the database that I update whenever a
> user marks an item as needing attention in my UI.  The problem I have
> is that I want to offer the ability to include need_attention in my
> user's queries, but do not want to incur the expense of having to
> reindex whenever this flag changes on an individual document.
>
> I have thought about different solutions to this problem, including
> using multi-core and having a smaller core for recently-marked items
> that I am willing to do 'near-real-time' commits on.  Are there are
> any common solutions to this problem, which I have to imagine is
> common in this community?
>