Question about recommender database drivers

[Matt Mitchell <go...@gmail.com>]

Hi,

The data I'm using to generate preferences happens to be in a solr
index. Would it be feasible, or make any sense, to write an adapter so
that I can use solr to store the preferences as well? The solr
instance could be embedded since this is all java, and would probably
end up being pretty quick. Our data is coming in fast, and I think
we'll outgrow the file based approach quickly. Thoughts?

- Matt

Re: Question about recommender database drivers

[kiran kumar <ki...@gmail.com>]

I have used the same steps to create the dictionary and vector output from
solr using *lucene.vector* command.
Is there any way to pull only latest changes from solr and create vectors.
Later how  do we run clustering algorithms using this incremented vector
files. Can you shed some light on this?

Thanks,
Kiran Bushireddy.

On Thu, Aug 2, 2012 at 3:04 AM, Sean Owen <sr...@gmail.com> wrote:

> The backing store doesn't matter much, in the sense that using it for
> real-time computation needs it to all end up in memory anyway. It can live
> wherever you want before that, like Solr. It's not going to be feasible to
> run anything in real-time off Solr or any other store. Yes the trick is to
> use Solr to figure out what has changed efficiently much like update files.
>
> If you're using Hadoop, same answer mostly. It's going to read serially
> from wherever the data is and most stores are fine at listing out all data
> sequentially.
>
>
> On Thu, Aug 2, 2012 at 3:52 AM, Matt Mitchell <go...@gmail.com> wrote:
>
> > Hi,
> >
> > The data I'm using to generate preferences happens to be in a solr
> > index. Would it be feasible, or make any sense, to write an adapter so
> > that I can use solr to store the preferences as well? The solr
> > instance could be embedded since this is all java, and would probably
> > end up being pretty quick. Our data is coming in fast, and I think
> > we'll outgrow the file based approach quickly. Thoughts?
> >
> > - Matt
> >
>



-- 
Thanks & Regards,
Kiran Kumar

Re: Question about recommender database drivers

[Sean Owen <sr...@gmail.com>]

The backing store doesn't matter much, in the sense that using it for
real-time computation needs it to all end up in memory anyway. It can live
wherever you want before that, like Solr. It's not going to be feasible to
run anything in real-time off Solr or any other store. Yes the trick is to
use Solr to figure out what has changed efficiently much like update files.

If you're using Hadoop, same answer mostly. It's going to read serially
from wherever the data is and most stores are fine at listing out all data
sequentially.


On Thu, Aug 2, 2012 at 3:52 AM, Matt Mitchell <go...@gmail.com> wrote:

> Hi,
>
> The data I'm using to generate preferences happens to be in a solr
> index. Would it be feasible, or make any sense, to write an adapter so
> that I can use solr to store the preferences as well? The solr
> instance could be embedded since this is all java, and would probably
> end up being pretty quick. Our data is coming in fast, and I think
> we'll outgrow the file based approach quickly. Thoughts?
>
> - Matt
>