dealing with so many different sorting options

[Jason Toy <ja...@gmail.com>]

As I'm using solr more and more, I'm finding that I need to do searches and
then order by new criteria.  So I am constantly add new fields into solr
 and then reindexing everything.

I want to know if adding in all this data into solr is the normal way to
deal with sorting.  I'm finding that I have almost a whole copy of my
database in solr.

Should I be pulling out all the data from solr and then sort in my database?
 This solution seems like it would take too long.
Could/Should I just move to solr as my primary store so I can query directly
against it without having to reindex all the time?


Right now we store about 50 million docs, but the size is growing pretty
fast and it is a pain to reindex everything everytime I add a new column to
sort by.

Re: dealing with so many different sorting options

[Erick Erickson <er...@gmail.com>]

Well, you're kind of stuck unfortunately. It's pretty much required that you'll
have to reindex when you add new fields if you want existing documents to
have that field.

I don't think there's any good way to use the DB to sort Solr results that would
be performant.

About using Solr as your data source. I'm not sure what you mean here. Solr
is many things, but it's not intended to be a data store. You can essentially
store the entire DB in the Solr index though, there's nothing wrong with that.
Admittedly, re-indexing is a pain but I suspect that as your app matures you'll
find yourself adding fields less and less often.

Sorry I can't offer better suggestions, but that's the nature of the developing
apps <G>..

Best
Erick

On Fri, Jul 29, 2011 at 3:36 PM, Jason Toy <ja...@gmail.com> wrote:
> As I'm using solr more and more, I'm finding that I need to do searches and
> then order by new criteria.  So I am constantly add new fields into solr
>  and then reindexing everything.
>
> I want to know if adding in all this data into solr is the normal way to
> deal with sorting.  I'm finding that I have almost a whole copy of my
> database in solr.
>
> Should I be pulling out all the data from solr and then sort in my database?
>  This solution seems like it would take too long.
> Could/Should I just move to solr as my primary store so I can query directly
> against it without having to reindex all the time?
>
>
> Right now we store about 50 million docs, but the size is growing pretty
> fast and it is a pain to reindex everything everytime I add a new column to
> sort by.
>